Mig release 5.7.0 rc8#10
Draft
mhaque07 wants to merge 371 commits intoKB-iGOT:mig-cbrelease-4.8.16_RC6from
Draft
Mig release 5.7.0 rc8#10mhaque07 wants to merge 371 commits intoKB-iGOT:mig-cbrelease-4.8.16_RC6from
mhaque07 wants to merge 371 commits intoKB-iGOT:mig-cbrelease-4.8.16_RC6from
Conversation
…thosh-tg/release-5.0.0 Release 5.0.0 - GCP related changes
…akartheekreddy/csp-migration Issue #KN-603 fix: CSP changes
…ect-sunbird/csp-migration Issue #KN-603 merge: Merge CSP migration change to release-5.1.0
This reverts commit 0275947.
…akhil46/release-5.0.0 Azure CSP changes
Signed-off-by: Keshav Prasad <keshavprasadms@gmail.com>
…shavprasadms/release-5.1.0 fix: SB-31155 using cloud_storage_url instead of hard coded url
# Conflicts: # kubernetes/ansible/roles/flink-jobs-deploy/defaults/main.yml
…akhil46/release-5.1.0 Added ES related roles
…eject and review APIs" This reverts commit bdb56f6.
…e-5.6.0 Issue KN-10 fix: disable content publish, unlisted publish, reject and review APIs
…prakash8887/release-5.4.1 Issue #KN-889 fix: QR ImageURL bulk sync
Merge Release 5.4.1 into 5.5.0
Merge Release 5.5.0 into 5.6.0
Issue #IQ-493 feat: added code for quml migration event generation
* Add task to start neo4j * merge Release 5.5.0 into 5.6.0 (Sunbird-Knowlg#1971) * Issue #KN-427 feat: Knowledge service cloud-agnostic * added artifact upload for oci oss storage Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * owerwrite oss object if already exists Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing with artifact download for neo4j from oci oss Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * changed the oss object download command Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added checkpoint Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed checkpoint for debuging Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added a cloud storage type selector for oci Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * jinja indentation Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * jinja indentation correction Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed the extra line for cloud_storage_type Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added s3 credential configuration for flink Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * disabled s3 storage temporarily Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * s3 config for flink jobmanager only Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * corrected if else syntax Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added s3 config for taskmanager Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added base url configuration for s3 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * jinja templating syntax error correction Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * corrected base url for s3 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated flink-conf for questionset-publish Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing with s3 configuration Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * endpoint url changed home region Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated values.j2 cloud_service_provider cloud_storage_endpoint Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added closing (") for variable cloud_service_provider Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added hadoop-fs parameters to flink-conf for questionset Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * corrected the flink-conf for questionset Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated presto specific variables in flink-conf.yaml Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added s3-fs-hadoop values in flink-conf for question set Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated the correct values for flink-conf Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed values form flink-conf.yaml Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated the deployment template Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added hive.s3.path-style-access Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed hive access path Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * template value for path.style Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added s3.aws.cred provider Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed s3.aws.credentials.provider Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing with flink-conf.yaml Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added s3.aws.credentials.provider Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated value of s3.aws.credentials.provider Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed s3.aws.credentials.provider Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added s3.aws.credentials.provider for asset-enrichment Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added acces key for asset enrichment Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added provider in the jobmanager deployment Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed aws provider from deployment and asset-enrichment Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added aws cred provider in job mgr deployment Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing with signing algo Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing with S3SignerType Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * update AWS3SignerType Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed signing algorithm Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * trying with s3a Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * reverted to s3 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * AWS4SignerType algo Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * AWS4UnsignedPayloadSignerType Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * NoOpSignerType algo Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * S3SignerType Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed signer algo Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added aws logging Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added http logging Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added flink debug logs Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added http logging Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed http logging Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing bucket probing Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * bucketprobe=2 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * bucketprobe=1 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * disable bucket probe Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed bucket probe Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing with disabing bucket probe Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * passing the region to hadoop configuration Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * presto config Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added the region to deployment Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated taskmanager with presto variables Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed oss connection details removed from flink-conf.yaml for questionset-publish as it no longer required Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * Added oci cloud-store-sdk updated artifact id to cloud-store-sdk_2.11 version: 1.45 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * cloud-store-sdk with exclusion Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed exlusion from cloud-store-sdk dependency Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * rearranged modules for service Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * reverted modules ordering Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added exclusion for guava for cloud-store-sdk Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed exlusion for guava Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated cassandra driver version to 3.2.0 This is test guava 20 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * update cloud-store-sdk version to 1.4.6 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * passing an empty region in cloud-store-sdk 1.4.6 region is defined as an Option<String> , so need to pass it as a optional string Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated cassandra driver to 3.1.2 cloud-store-sdk 1.4.6 uses guava 19, so reverted the change from 3.2.0 Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * Updated the oss_bucket_name * Updated the instance principal * Added env var for OCI * Corrected the oss bucket name key * added oci specs for video-stream deployment for kp flinkjobs Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated the zookeeper ip from local host to the ingestion and processing Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * disabled service_fact gathering Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * disabled service_facts Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * Update values.j2 To add the media workflow id * Issue #KN-0000 feat: Config update * Issue #KN-0000 feat: Config update * Issue #KN-0000 feat: Config update * Issue #KN-0000 feat: Config update * added endpoint to flinkjob configuration Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added the alter replication and partition Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing list length condition Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added condition to create topic Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * removed replication factor alter Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * oci media streaming changes * added cloud_storage_type key Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * udpated cloud-store-sdk id Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * resolved conflicts Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added cloud_storage_proxy_host for qrcode job Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added value for proxy host Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * Issue #KN-195 feat: CSP migrator parallelism increase * To troubleshoot download content issue * Adding the missing property * Reverting the changes made in this deploy branch * missing property * for testing publish unit test Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated the continer name value for unit test case Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing unit tests Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated endpoint for unit testing Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * youtube chcks Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * for testing unit case with azure Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing unit cse Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * added cloud ep for unit test Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * adding the missing config * updated generic variables Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * for test runs in circleci Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated test data Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated youtube link Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated invalid youtube links Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * updated youtube link for test cases Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * testing creative commons lic Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * passing mime type field for test case Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * reverted the mimetype value Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> * KN-CSP Changes * KN-CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * KN-Fix For CSP Changes * Issue #KN-920 fix: Update kubernetes/ansible/roles/flink-jobs-deploy/defaults/main.yml Co-authored-by: Mahesh Kumar Gangula <mahesh@sanketika.in> * Revert "KN-CSP Changes" * Issue KN-921 fix: config issue fix * Issue KN-921 fix: config issue fix * Issue KN-921 fix: config issue fix * Issue KN-921 fix: AWS support for Flink jobs --------- Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> Co-authored-by: Jayaprakash8887 <Jayaprakash.narayanaswamy@tarento.com> Co-authored-by: vinukumar-vs <vinukumar_vs@yahoo.com> Co-authored-by: Deepak Devadathan <deepak.devadathan@oracle.com> Co-authored-by: Kenneth Heung <kenneth.heung@gmail.com> Co-authored-by: subhash_chandra_budde <subhash.chandra.budde@oracle.com> Co-authored-by: rjanart <123344037+rjanart@users.noreply.github.com> Co-authored-by: Anil Gupta <ani2824@gmail.com> Co-authored-by: Ramya <ramya.janarthanan@oracle.com> Co-authored-by: Aiman Sharief <aiman.sharief.17@gmail.com> Co-authored-by: Mahesh Kumar Gangula <mahesh@sanketika.in> --------- Signed-off-by: Deepak Devadathan <deepak.devadathan@oracle.com> Co-authored-by: santhosh-tg <santhoshkumar.sreedharan@tarento.com> Co-authored-by: Jayaprakash8887 <Jayaprakash.narayanaswamy@tarento.com> Co-authored-by: vinukumar-vs <vinukumar_vs@yahoo.com> Co-authored-by: Deepak Devadathan <deepak.devadathan@oracle.com> Co-authored-by: Kenneth Heung <kenneth.heung@gmail.com> Co-authored-by: subhash_chandra_budde <subhash.chandra.budde@oracle.com> Co-authored-by: rjanart <123344037+rjanart@users.noreply.github.com> Co-authored-by: Anil Gupta <ani2824@gmail.com> Co-authored-by: Ramya <ramya.janarthanan@oracle.com> Co-authored-by: Aiman Sharief <aiman.sharief.17@gmail.com> Co-authored-by: Mahesh Kumar Gangula <mahesh@sanketika.in>
Issue #IQ-670 fix: fix for selective quml migration
…e-5.7.0 Release 5.7.0
Issue #KN-1089 fix: added default value for endpoint variable
Issue #KN-1089 fix: added default value for proxy host
…n-event-processor transaction-event-processor config added
…n-event-processor transaction-event-processor config added
…mediakind Issue KN-1096 feat: Video streaming job MediaKind configurations added
Issue KN-1096 feat: Removed unused AMS variables
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.